Crowdsourcing Interactions A proposal for capturing user interactions through crowdsourcing
نویسندگان
چکیده
Crowdsourcing has been proposed as an inexpensive and often efficient way of outsourcing work tasks to a large group of people. In this paper, we propose to use crowdsourcing as strategy for acquiring users interactions within interactive information retrieval (IIR) systems. We are interested to understand whether crowdsourcing represents a robust strategy that can be used in conjunction with common approaches for capturing interactions, in particular laboratory user experiments. What are the similarities, differences, advantages and disadvantages of crowdsourcing interactions compared to traditional strategies? To investigate these issues, we outline the design of a procedure where interactions can be captured using crowdsourced workers. We expose the problematic issues that arise during the design process, together with preliminary statistics and results acquired by implementing our protocol within Amazon Mechanical Turk. This work opens up a number of research prospectives, the most appealing being a new methodology for the evaluation of IIR systems based on crowdsourcing.
منابع مشابه
Enabling Trust in Crowd Labor Relations through Identity Sharing
While online Crowdsourcing marketplaces provide a powerful avenue for facilitating new forms of informationdriven micro-labor, their practical value is significantly reduced by worker “spam” and employer fraud. We hypothesize anonymity of parties is a major source of these problems, and we thus propose a human-centric solution: encourage employers and workers to voluntarily deanonymize in order...
متن کاملPerform Three Data Mining Tasks with Crowdsourcing Process
For data mining studies, because of the complexity of doing feature selection process in tasks by hand, we need to send some of labeling to the workers with crowdsourcing activities. The process of outsourcing data mining tasks to users is often handled by software systems without enough knowledge of the age or geography of the users' residence. Uncertainty about the performance of virtual user...
متن کاملCustomized crowds and active learning to improve classification
Traditional classification algorithms can be limited in their performance when a specific user is targeted. User preferences, e.g. in recommendation systems, constitute a challenge for learning algorithms. Additionally, in recent years user’s interaction through crowdsourcing has drawn significant interest, although its use in learning settings is still underused. In this work we focus on an ac...
متن کاملIncentive Mechanisms for Crowdsourcing Platforms
Crowdsourcing emerged with the development of Web 2.0 technologies as a distributed online practice that harnesses the collective aptitudes and skills of the crowd in order to reach specific goals. The success of crowdsourcing systems is influenced by the users’ levels of participation and interactions on the platform. Therefore, there is a need for the incorporation of appropriate incentive me...
متن کاملService Oriented Protocols for Human Computation
Human computation and crowdsourcing are increasingly gaining momentum. Many platforms already exist providing basic features for crowdsourcing different types of tasks to the Web. Service Oriented Architectures (SOA) provide the ideal technical framework to support interactions with both Human-Provided Services (HPS) and Software-Based Services (SBS). A unified service-oriented computing approa...
متن کامل